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Problem-based learning (PBL) is defined as a student- centered pedagogy which can provide learners more 
opportunities for application of knowledge acquired from basic science to the working situations than 
traditional lecture-based learning (LBL) method. In China, PBL is increasingly popular among preventive 
medicine educators, and multiple studies have investigated the effectiveness of PBL pedagogy in preventive 
medicine education. A pooled analysis based on 1 5 studies was performed to obtain an overall estimate of the 
effectiveness of PBL on learning outcomes of preventive medicine. Overall, PBL was associated with a 
significant increase in students' theoretical examination scores (SMD — 0.62, 95% CI — 0.41-0.83) than 
LBL. For the attitude- and skill-based outcomes, the pooled PBL effects were also significant among learning 
attitude (OR - 3.62, 95% CI - 2.40-5.16), problem solved skill (OR - 4.80, 95% CI - 2.01-11.46), 
self-directed learning skill (OR - 5.81, 95% CI - 3.11-10.85), and collaborative skill (OR - 4.21, 95% CI = 
0.96-18.45). Sensitivity analysis showed that the exclusion of a single study did not influence the estimation. 
Our results suggest that PBL of preventive medicine education in China appears to be more effective than 
LBL in improving knowledge, attitude and skills. 

Problem-based learning (PBL) was first implemented at McMaster University Medical School in the 1960s 
where it has revolutionized the field of medical education since then'. PBL is defined as a student-centered 
pedagogy in which participants are allocated in groups of up to 8 persons under non-directive tutors and 
given tasks or challenges that reflect situations relevant to the working environments they wUl experience^. In this 
way, the learners are empowered to integrate theory and practice, and apply knowledge and skills to develop a 
viable solution to the problem'"'''. PBL represents a paradigm shift from traditional pedagogy, which is more often 
lecture-based learning (LBL). LBL pedagogy focuses on factual knowledge and memorization, providing little 
chance for application of knowledge acquired from basic science to the working situation. There are many other 
advantages of PBL over LBL, including flexible knowledge, improved communication, collaborative skills and 
self-directed learning skills, and a more enjoyable and motivational format^. 

Originally, PBL was devised in an attempt to develop a teaching method for use for clinical medical education. 
A variety of medical schools in Europe (the Netherlands, Sweden, Norway, and the United Kingdom), Asia 
(Japan, Korea, and China), Canada, the United States, South Africa, Australia and New Zealand already have or 
are introducing PBL as a learning modality' *' Now it has also been used in multiple disciplines including public 
health and preventive medicine'. For example. Dyke et al. had used PBL approach for teaching epidemiology, and 
found that the PBL students got a far richer learning experience than their counterparts in the traditional stream'". 
As the public health system in China has been developing rapidly for the past decade, PBL is increasingly popular 
among preventive medicine educators striving to produce graduates who can solve complex public health 
problems efficiently. In recent years, a growing number of Chinese studies have investigated the effectiveness 
of PBL pedagogy in preventive medicine education. As one of the most well-known public health schools in 
China, the School of Public Health from Nanjing Medical University (NMU) has adopted a PBL in preventive 
medicine education from September 2012. NMU has established multidisciplinary cases throughout the course 
schedule to improve the positive outcomes, such as motivating learning, problem solved skill, collaborative skill 
and critical thinking. To the best of our knowledge, however, there are no reports or reviews summarizing the 
trials done to evaluate the effectiveness of PBL for preventive medicine education. 
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Records identified through database searching 
(n= 110) 



Excluded after title/abstract review (n ^ 46) 



Identified as appropriate studies for inclusion 
(n - 64) 



Excluded (n = 52) 

Review or editorial (n^23) 

No relevant quantitative outcomes (n ^ 16) 

No comparison with lecture-based learning (n = 9) 

Duplicate publication (n = 1) 

No full text (n = 3) 



Articles included 
{n- 12) 



Additional articles included from reference review 
(n-3) 



Studies included with adequate data 
(n=15) 



Figure 1 | Flow chart summarizing identification of included studies. 

The uncertainty about the effectiveness of PBL and the heterogen- 
eity in the pubUshed literatures provided the impetus for this study. 
Thus, the aim of the study was to perform a meta-analysis on all 
published studies to obtain an overall estimate of the effectiveness of 
PBL on learning outcomes of preventive medicine education in 
China. 

Results 

Study characteristics, and quality assessment. The study selection 
process is depicted in Fig. 1. The search strategy yielded 110 entries, 
of which 64 in Chinese were considered to have potential value and 
were subjected to further examination. Based on the inclusion and 
exclusion criteria, we excluded 52 articles (23 were reviews or 
editorials; 16 no relevant quantitative outcomes; 9 no comparison 
with LBL; 3 no full text and 1 duplicate publication). Three additional 
articles were included from the reference review. Ultimately, a total of 
15 articles involving 2,061 subjects published from 2006 through 



2013 were included in this meta-analysis""^^. Characteristics of the 
selected studies are presented in Table 1. 

The methodological quality of each included study is summarized 
in Table 2. The quality scores ranged from 4 to 6. A majority of the 
studies are cluster randomized trials, while only one is randomized 
controlled trial (RCT). Blinding is a critical measure to protect 
against bias, however, none of the studies mentioned the application 
of this method. Some other bias protection measures such as control 
for important factors and incomplete data bias were often used in the 
publications. Whether the outcome assessors and data collectors 
were blinded to subjects' assignments or not was not mentioned in 
the studies. 

Quantitative synthesis. The effects of PBL methods were evaluated 
by synthesizing theoretical examination scores, pass rate, the 
improvement on learning attitude, problem solved skill, self- 
directed learning skill and collaborative skill in this meta-analysis. 
The effects of PBL on theoretical examination scores were reported 
by all the included studies. When all the eligible data were pooled into 
the meta-analysis, PBL was associated with a significant increase in 
students' examination scores (SMD = 0.62, 95% CI = 0.41-0.83, 
J'heterogeneity < 0.001) (Fig. 2). Four studlcs reported pass rate of the 
theoretical test, when they were pooled together, there was a lack of 
heterogeneity (P = 0.83) and the PBL effect was significant (OR = 
2.88, 95% CI = 1.44-5.75). For the attitude- and skills-based 
outcomes, the pooled PBL effects were also significant (OR = 3.62, 
95% CI = 2.40-5.16, Pheterogeneity = 0.444 for learning attitude; OR = 
4.80, 95% CI = 2.01-11.46, Pheterogeneity < 0.001 for problem solved 
skill; OR = 5.81, 95% CI = 3.11-10.85, Pheterogeneity = 0.048 for self- 
directed learning skiU; and OR = 4.21, 95% CI = 0.96-18.45, 
-Pheterogeneity < 0.001 for coUaborativc skiU) (Table 3). 

We then evaluated the effects of PBL in students' examination 
scores according to the quality scores. For quality scores = 6, SMD 
= 0.70, 95% CI = 0.37-1.04, Pheterogeneity < 0.001; scores = 5, SMD 
= 0.53, 95% CI = 0.37-0.70, Pheterogeneity = 0.153; scores = 4, SMD 
= 0.38, 95% CI = 0.03-0.73, Pheterogeneity = 0.924. The result indi- 
cated that studies with high quality scores would yield significantly 
larger effect sizes than those with low quality scores. 



Table 1 I Characteristics of included studies 



No. of participants 



Study 


Year 


Disciplines or curricula 


PBL 


LBL 


Source of participants 


Outcome assessment 


Zliangjieetal. 


2013 


Environmental health 


54 


50 


Rural doctors in Han Chinese 


Examination scores and questionnaire surveys 






55 


61 


Rural doctors in minority 


Examination scores and questionnaire surveys 












nationalities 




Li Guongyou et al. 


2013 


Medical statistics 


45 


48 


Undergraduate students 


Examination scores and questionnaire surveys 


Gao Lanyue et al. 


2013 


Occupational health and 
medicine 


61 


61 


Undergraduate students 


Examination scores 


Zliao Yingzheng etal. 


2013 


Toxicology 


40 


38 


Undergraduate students 


Examination scores and questionnaire surveys 


Zliang Zhiliong et al. 


2012 


Environmental health 


77 


57 


Undergraduate students 


Examination scores and questionnaire surveys 


Zhang Qin et al. 


2012 


Child and adolescent health 


50 


51 


Undergraduate students 


Examination scores and questionnaire surveys 


Wang Shuran et al. 


201 1 


Nutrition science and food 


10 


1 16 


Undergraduate students 


Examination scores and questionnaire surveys 






hygiene 










Lu Zhiquan et al. 


201 1 


Epidemiology 


150 


150 


Postgraduate students 


Examination scores and questionnaire surveys 


Hu Dongmei et al. 


201 1 


Medical statistics 


90 


90 


Postgraduate students 


Examination scores and questionnaire surveys 


Sun Rong et al. 


2010 


Medical statistics 


86 


85 


Undergraduate students 


Examination scores and questionnaire surveys 


Zhang Yanqi et al. 


2010 


Medical statistics 


19 


39 


Eight-year program medical 


Examination scores and questionnaire surveys 












students 




Zhang Yanqi et al. 


2010 


Medical statistics 


39 


39 


Eight-year program medical 


Examination scores and questionnaire surveys 












students 




Wu Songwen etal. 


2010 


Medical statistics 


60 


60 


Undergraduate students 


Examination scores and questionnaire surveys 


Zhang Yixi et al. 


2009 


Medical statistics 


93 


78 


Higher vocational school 


Examination scores and questionnaire surveys 










students 


Deng Shusong et al. 


2006 


Preventive medicine 


53 


56 


Undergraduate students 


Examination scores and questionnaire surveys 



PBL, problem-based learning; LBL, lecture-based learning. 
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Table 2 | Methodologic quality of studies included in the meta-analysis 






Allocation 


Control for important 


Control for 


Assessment of 


Total quality 


Study 


Year Randomization 


concealtnetnt Blind 




factors" 


incomplete data bias outcome*" 


scores 


Z.nQnQ Ji6 ©t" qI. 




A 




AA 


A 


AA 


A 

o 


LI wUUrlUyUU cJI (Jl. 


201 3 


A 




A 


A 


AA 


5 


Gao Lanyue et al. 


ZU 1 O 


A 




AA 


A 


A 




/hnri Yinri7npnn pt n 


2013 


A 




AA 


A 


AA 


6 


Z_rliJllLJ £_l 1 1 1 li^r ILJ d Ul. 


201 2 


A 




AA 


A 


AA 




/nnnfi (win ot nl 

Z-IIUIILJ 1 1 1 CI IJI. 


201 2 A 


A 




A 


A 


AA 


5 


^A^Q^Q Shuron q\ qI. 


901 1 






A A 


A 


AA 


i; 


Lu Zhiquan et al. 


201 1 


A 




AA 


A 


AA 


6 


Hu Dongmei et al. 


201 1 


A 




AA 




AA 


5 


Sun Rong et al. 


2010 


A 




AA 




AA 


5 


Zhang Yanqi et al. 


2010 


A 






A 


AA 


4 


Zhang Yanqi et al. 


2010 


A 






A 


AA 


4 


Wu Songwen et al. 


2010 


A 




AA 


A 


AA 


6 


Zhang Yixi et al. 


2009 


A 




AA 


A 


AA 


6 


Deng Shusong et al. 


2006 


A 




AA 


A 


AA 


6 


"A maximum of 2 triangles couid be awarded for this item. Studies 


that controlled for age received 


triangle, controlled for previ 


Dus academic performance 


received an additional triangle. 


''A maximum of 2 triangles could be awarded for this item. Studies 


that measured by examination scores or questionnaire surveys 


received 1 triangle, measured by both examination 


scores and 


questionnaire surveys receive 


d two triangles. 















Test of heterogeneity. There was significant heterogeneity (P < 
0.001) among included studies. We assessed the source of 
heterogeneity by quality scores, source of participants and sample 
size (participants more than 50 in both PBL and LBL groups). As a 
result, quality scores (P = 0.035) and source of participants (P = 
0.048), but not sample size (P = 0.146), were found to contribute to 
the substantial heterogeneity. 

Sensitivity analyses. Sensitivity analyses indicated that Lu's study 
was the main origin of heterogeneity. However, when this article was 
excluded the overall resuh (SMD = 0.57, 95% CI = 0.38-0.77, P < 
0.001) and the heterogeneity (P < 0.001, P = 72.6%) was not 
materially influenced. Further exclusion of any single study got 
simOar results with a range from 0.56 (95% CI = 0.37-0.75) to 
0.67 (95% CI = 0.46-0.87), suggesting that the results of this 
meta-analysis are statistically reliable. 

Publication bias. Begg's funnel plot and Egger's test were performed 
to evaluate the publication bias of the literatures. As a result, the 
funnel plots did not show any obvious asymmetry. Then, the 
Egger's test was adopted to provide statistical evidence of funnel 



Zhang Jie (1) et 
Zhang Jie (2) et 
Li Guangyou et 
Gao Lanyue et 
Zhao Yingzheng et 
Zhang Zhihong et 
Zhang Qin et 
Wang Shuran et 
Lu Zhiquan et 
Hu Dongmei et 
Sun Rong et 
Zhang Yanqi (1) et 
Zhang Yanqi (2) et 
Wu Songwen et 
Zhang Yixi et 
Deng Shusong et 
Overall (l-square 



2013' 
2013' 
2013' 
2013' 
2013' 
2012' 
2012' 
201 r 
2011 
201 r 
2010' 
2010' 
2010' 
2010' 
2009' 
2006' 



= 80.1%)- 



-0.5 0.0 0.5 1.0 1.5 2.0 2.5 
SMD (95% CI) 



Figure 2 | Forest plot for the effects of PBL on examination scores 
compared with LBL. 



plot symmetry. The results also indicated no evidence of publica- 
tion bias (P = 0.432). 

Discussion 

Many medical schools internationally are changing their curricula 
and moving to PBL programs^''. While in China, the application of 
PBL pedagogy is still in the initial stage, especially among preventive 
medicine education. In this meta-analysis, we found that compared 
with LBL, PBL was more effective in all domains including theor- 
etical knowledge, attitude and sldlls. Nevertheless, the results of atti- 
tudes and sldlls were clearly less precise than that of the theoretical 
knowledge. It may be due to the less eligible studies and smaller 
sample sizes. Moreover, the assessment of theoretical knowledge 
by a test or exam seems more objective than that of attitudes and 
skills by the participants' sensation. 

PBL is one of the best described interactive learning methods^'. 
According to the questionnaire results and the tutors, students 
showed more enthusiasm for PBL rather than LBL. It is reported 
that learning is most effective when students are actively involved 
in PBL^**. Therefore, our results were consistent with previous con- 
clusions, which may improve the application of lesion-based theory 
to practical knowledge as well as achieving specific goals. Although 
the heterogeneity amongst studies was large (F = 80.1%), we found 
that the sources of heterogeneity were from quality scores and source 
of participants. Furthermore, the results of the subgroup analysis 
only partially explained the heterogeneity. Variations in other 
aspects such as study design, involved discipline, and the duration 
may also play a part. 

In the analysis stratified by quality scores, elevated effects were 
more pronounced among studies with higher quality score, suggesting 
the importance of methodological quality during PBL intervention. 
Overall, the methodological quality of the included studies in this 
meta-analysis was not high. First, most of the studies are not rando- 
mized controlled trials (RCTs). In China, students are divided into 
classes when they entered the university, so the researchers prefer to 
allocate according to the classes rather than the individuals. Second, as 
an investigation in the field of education, it is impossible for the 
researchers to use the blinding method during the whole curriculum. 
Third, there is no standard criterion for evaluating the effectiveness of 
the PBL pedagogy. Whether the outcome assessors and data collectors 
were blinded to subjects' assignments or not was also poorly reported. 
In a word, high risks for selection bias, performance bias, as well as 
measurement bias do exist in the literatures we included. 
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Table 3 | Summary of effect sizes for PBL and LBL 



Outcome 


No. of studies 


No. of subjects 


SMD/OR (95%CI)'' 


F 


P for heterogeneity 


Examination scores 


15 


2061 


0.62 (0.41-0.83) 


80. 1 % 


<0.001 


Pass rate of examination 


4 


495 


2.88 (1.44-5.75) 


0.0% 


0.833 


Questionnaire surveys 












Learning attitude 


4 


546 


3.62 (2.40-5.16) 


0.0% 


0.444 


Problem solved skill 


6 


851 


4.80 (2.01-11.46) 


82.6% 


<0.001 


Self-directed learning skill 


5 


690 


5.81 (3.1 1-10.85) 


58.3% 


0.048 


Collaborative skill 


3 


426 


4.21 (0.96-18.45) 


87.5% 


<0.001 



"Random-effects model was used when Pvolue for heterogeneity test <0. 1 0; otherwise, fix-effects model was used. 

PBL, problem-based learning; LBL, lecture-based learning; SMD, standardized mean difference; OR, odds ratio; CI, confidence interval. 



In conclusion, our meta-analysis shows that PBL in preventive 
medicine education appears to be more effective than LBL in improv- 
ing knowledge, attitude and sldlls. For future studies assessing the 
implementation of PBL, high quality of problems, uniform and 
objective outcome measurement, and well-designed RCTs are 
required. Only through studies which are strategic planned and con- 
ducted with carefully quality controlled, can we ever expect to 
achieve the practical goals. 

Methods 

Search strategy. Relevant biomedical sciences and educational databases [PubMed, 
EMBASE, China National Knowledge Infrastructure (CNKI), and Chinese 
Biomedical Literature database (CBM) were searched from inception through 
September 201 3. The search strategy was to use and combine the following key words: 
"problem-based learning", "PBL", "preventive medicine", "epidemiology", "medical 
statistics", "health statistics", "environmental health", "nutrition science and food 
hygiene", "occupational health and medicine", "child and adolescent health", 
"toxicology", "hygienic toxicology", or "social medicine". In addition, references of 
all included articles were also identified by a manual search and studies matching the 
eligible criteria were retrieved. 

Inclusion criteria. Studies included in the current meta-analysis have to meet the 
following inclusion criteria: (1) the study should compare the effectiveness of PBL 
group and traditional LBL group; (2) the courses of PBL pedagogy should be 
preventive medicine professional disciplines; (3) both randomized and non- 
randomized studies were considered, and (4) quantitative data about the effectiveness 
of PBL were available (i.e., they had to report a mean change in knowledge, attitudes 
or skills that occurred in response to an PBL intervention and was measured using a 
numeric scale such as an examination or test). 

Data extraction and quality assessment. For each study, the following basic 
information was collected: the first author's name, publication year, the involved 
disciplines, number of participants in each group, educational background of the 
participants, mean scores of both groups and the outcome measure. Two of the 
authors extracted all data independently according to the inclusion criteria listed 
above and reached a consensus on all the items. If more than one article was published 
using the same population, we selected the most recent or most informative report. 

Recognizing that many nonrandomized studies would be included, an adaptation 
of the Newcastle- Ottawa Scale was adopted to abstract information on methodo- 
logical quality. The included studies were judged in terms of a proper method of 
randomization, allocation concealment, blinding, the comparability of the popula- 
tions, the completeness of the data, and the assessment of outcome. The full score was 
8 triangles, and a high-quality study was defined as a study with 5 or more triangles. 

Statistical analysis. Standardized mean difference (SMD) for continuous data and 
odds ratio (OR) for dichotomous data were performed to estimate the pooled effects. 
Heterogeneity assumption was identified by the P statistics. A P-value < 0.10 for the 
F test indicated a lack of heterogeneity among the studies, and then random -effects 
model (DerSimonian and Laird method) was used to calculate the summary 
standardized mean difference (SMD) or OR estimate of each study^^. Otherwise, the 
fixed-effects model (the Mantel-Haenszel method) was used^". Sensitivity analyses 
were performed to assess the stability of the results, namely, a single study in the meta- 
analysis was deleted each time to reflect the influence of the individual data set to the 
pooled SMD. To evaluate the publication bias. Funnel plots and Egger's linear 
regression test was applied". AU analyses were carried out with Stata software 
(version 11.0; StataCorp LP, College Station, TX, USA), using two-sided P values. 
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